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ABSTRACT v 

Maryland's School Improvement Through Instructional 
Process (SITIP) program involves all 24 local education agencies in 
implementing one ^or more of the following research-based 
instructional models: Active Teaching, Mastery Learning, Student Team 
Learning, and Teaching Variables. Following a review of background 
information, state education agency initiatives and assistance 
activities are described, an evaluative overview is presented, and 
local impact is discussed. The state initiatives consisted of 
planning, training activities, and technical assistance to local 
education agencies. Planning was flexible, interactive, ongoing, and 
based on an open systems approach, resulting in a high degree of 
local participation and commitment to the program at nearly all 
sites . State-sponsored training activities included an assistant 
superintendent * s retreat, an instructional leadership conference, and 
followup workshops . Technical assistance was proyide<| by an ^ 
eight-person team that provided statewide leadership while 
encouraging local ownership and that developed networks and 
teaching/learning opportunities for local teams to share successes 
and build expertise. Impacts of the SITIP program on student 
achievement, student attitudes, teacher's knowledge of models, school 
environment, and administrators and cen'tral office staffs are 
reported, along with^oncerns and recommendations by local SITIP ' 
participants. (TE) . 



************************************** 

* Reproductions supjplied by EDRS are the best that can be made * 

* , from the original document. ^* 
*********************************************************************** 



ERLC 



Ihtroduction 



This paper.describes. the impact of a' statewide instructional improvethent 
program on local education agencies (LEAs) , and discusses coordination of 
evaluation >ti^6cedures. Following a reviet/ of background information, state 

■ ^ " . • / • ' . ^ 

educatioi^&atencV (SEk) initiatives and assistance activities are described. 



an evaluation overview is presented, and local^/mpact is discussed. ; 

■ ■ ■ • ■•; v'e^, ■ / ■ ^ ■ :■. 

- Background Information 

Maryland's School Improvement Through Instructional Process (SITIP) 
progrfJm encourages application of research on planned change to implement one, 
or mote of four researched-b'ased instructional -models. The Maryland State 
Pepartment of Education (MSDE) supports local implementation by providing^ 
funds,, training, and technical assistance, and by conducting evaluation with 
reports developed so that data-based decisions can be made.*^ The instruc- . 
tional models used in SITIP are: Active, Teaching, Mastery Learr ilig, Student 
Team Learning, and Teaching. Variables . All 24 local education agencies (LEAs) 
in the state voluntarily implemented one or more of the models. 

SITIP is a multi-year program consisting of interactive activities which 
are outlined below. - v 



■ . • ■ ■ 

1. Preparation (opep systems planning):. Identify needs and potential 
solutions. Draft a design. 

2. InjLtial Commitment : Review plan with LEA superintendents. \Get 
commitment for local team attendance at awar ene Ss conferences . 



* Three major evaluation reports w4.ll have been written by the tinre^^^ect 
involvement by MSPE com68 to an end.. .The fiyst focused on._implemenrktion 
for the period December 1980 to June 1982. See: Roberts, et al., Instruc- 
. tional improvement in Maryland: A study of research in practi ce, 1982. 
ERIC #: Fu^l report, Ep222486; ^ixecutive summary, ED223553, The last, 
whicli will .cover , the 1983-84, school year, will focus on in-stitutionallza- 
tion. 



Awareness Conferences ; Have each of the. instructional; improvement . ... 
tnodels presented by its developer at awareness conferences attended^, 
by LEA teams, "mSDS staff, .and interested ofeher-s.- Describe design^^d* 
nature of (voluntary team) involvement. " • ; ' 

■ ' . * ' • . ' ■ ' • ■ ■ 

Local Proposals/Plans ; "'Help\ cross-hierarchical local team.s draft 
proposals to implement- one or more of the models. ; . 

Implementation ; Help L'EAs implement .'selectea-.models using' their p^n • 
strategies but^ involving representatives of all role groups.. 

Dissemination ; Encourage Use of '-the. models in many schools, and 
share infomlait ion about successes between LEAs. ^ 

7. Technical Assistance ; Assign MSDE»stkff (adross divisions) to assist 
LEAs in planning, implementation, and dissemination; to conduct 

■ follow-ups; and to facilitate networking . Build capacity; do not 
create dependency. " ' • • , . 

8. Follow-up Training ; Conduct an intensive three-day. training ' 
session on each, model for prospective implementers (teachers, 

' school administ|at6rs, central office- staff); vConduct annual or 
'• bi-annual follo«-up training sessions (using participatory planning) 
to maintain quality implementation. Assist LEAs (centr-al office 
, .'staff) in planning/conducting turnkey' training. ! - . 

9. Assessments of Progress 'and Impact (cyclic); Have a ."third party 
eyaluator" collect and analyze data systematically and us-e (feed- 
back) ^information to make improvements and publicize successes. 

•The activities outlined above began in 1980. ^ All'- 24 LEAs were.Vrepre- 

sented at orientation conferences. "T^inetee^ . LE\s submitted'pfoposals for , 

implementation -through June 1983. Fpy the;. 1982-84 school years, five "new'-' 



LEAs decided to. participate. - " 

■ ■ ■ ' / ' ■ ■ ■'• , ■ . 

• ' - / • -. The Instructional Models 

' Each of tlie instructional models is described here., 

~" Ac t Ive Teaching (AT) Ik oF "direct instruction devel- 

oped by Thomas Good and Douglas Grouws'^ at the University of ', 
Missouri. Orieinally designed tor tTie teaching of mathe- 
matics, XT consists of the following' components: (1) pre- , 
. lesson development — concepts and skills^from the previous 

' . niglit.^s homework are reviewed, homework is checked and 

collected, and students engage in mental* exercises; (2) lesson 
development ~ t)rerequi8ite skills and concepts are briefly 
reviewed, new concepts 'are introduced via teacher explanation 

' and demonstrati^on, and student comprehension, is assessed . 




through ^^rifr^Tlrtl rT^/^^^ iMHl W P — uninterrupted, 

individual, .Successful pl^HHBHJmrovid^d in order t;o 
in.crease .proficiency i^ thpllHKH taught; (4) 

' homewotk -"-^ homework is afesigriMMM 1 to the concepts 
developed that day; and (5)/rev^B^»interiance ~ weekly and 
6nd-of -unit reviews help to iriaiftt^^ skiils and concepts 
taught, V' - , . 

, ■ , < ^ ' " . •** 

Mastery Learning (ML), develop4dJ^ Benjamin Bloom and: James. 




Block, ; combines wrriculum aldfe^ilTOt and diagnostic/presfcrip- 
; tive instruction with .a^hilosD^|i^^that all students can 
succeed. Essential pomp^nents a«:' (1) developing a scope 
and sequence of /objectives, broken down into prerequisite and 
component skills; (2) providing appropriate instruction 
^ aligned- with the objectives to be mast'ered; (3) testing the 
V student *s progress in mastering th^ objectives through the use 
of a formative evaluation measure ("no- fault" test) ; (4) 
providing students who have not achieved mastery with addi- 
tional corrective work in the deficient areas specif ied- by the, 
formative tests, and providing students who have achieved 
mastery with enrichmeat activities \.p reinforce and supplement 
learning; (5) testing final maste'ryiof the objectives with a 
summative evaluation measure; and (6) recording student 
progress- in terms of individual mastery of specific ^ 
objectives.; "Mastery" is .usuaily defined as 80% of the 
students demonstrating success on at- least 8,0% of the 
obje,ctives in a given, unit of- instruction. ^\ ' 

Student Team Learning (STI,) techniques use peer tutprigg and ^ 
team competition to facilitate student learning. Student 
Team-Achievement Divisions (STAD) . and Teams-Games-Tournaments 
(TGT) were developed by Robert Slavin and staff at the Johns 
. Hopkins University. Jigsaw was started at the University of 
California at Santa Cruz. The key factors of STL are peer • 
interaction, Cooperation, and comi^tition. /STAD is basically 
team learning; TGT is team learning plus competition by 
ability level; Jigsaw is team learning of specific elements of 
a program, With regrouping' for peer teaching across elements. 

. • ■■ . . ■ I ■; 

Teaching Variables (TV) was developed by David, Helms and staff 
a^t Research for Better Schools (RBS) . Two variables found to 
be strongly related to effectiveness of Instruction and stu- 
dent achievement. were/dHentified:, "content" and "time." The 
"content" variable encompasses two factors: (1) assessment of 
prior learniTig^ and (2) aj.ignment of .curriculum objective's and 
classroom instruction to/the testing instrument. ^The "time" 
variable improvement .cycle involves: (1) measuring s tudent^en- 
gaged time (SET) via cias^room observation, (2) ^sbmparing SET . 
and opportunity^ for improvement , (3) rev±e;^ing and selecting 
research-based improvement strategies, (4) implementing 
strategies, and (5) using additional classroom observations to„ 
'evaluate the-ef fectiveness of the strategies in improving SET. 



"^^ State. Initiatives and Assistance 

Maryland State Department . of Education (MSDE) st^ff engaged in planning, 
designed and conducted a variety of training act;ivities, and provided 

technical assistance (TA) to LEAs, * ' . 

•' .■ * . * ■ ■ ■ / 

Planning ' » , / 

Planning was flexible, interactive, on-going', and based on an open- 

* : ■ • ■ ■ : ' ^ ^ ■ ■ . • ■ . V •• ■ 

systems approach. Existing organizational structures were used. or new ones 

. ■ " ■ . ■ ■ / • 

developed to 'facilitate communication and involve various interest^ groups in , 

MSDE-initiat^d plans. Within MSDE and between MSDE and the^ LEAs>, e^f forts, were 



made to cdordlnate activities and to .strengthen or integrate existing programs 



with (or, SITIP knowledge bases on instruction and planned change). 

Planning Waa' timely, made good use of resources «and available expertise, and 

. , ; - ' ■ ■ ■ ■ ■ / 

invited li>cal participation by role groups and in sucli ways as to result in 
high commitment td the program, and real (not lip-service) implementation in 
almost all sites. Major outcomes of planning activities within MSDE (other 

than the implementation of the plans) included: (1) a general knowledge of 

•• ' ' ' ■ . ' ■ ' ' ■ ' 

* ' ' ■ '■ 

STTIP by most MSDE staff; (2) sufficient commitment . or interest by senior and 
middle management to be willing to explore elements of knowledge basey of 
SITIP., and to' continue (and expand) cooperative support for technical, assisi:-.. 
ance; (3) applica-tion of SITIt— related inf ormatiou, strategies, or processes 
in various existing programs; and (4) increased knowledge and skills in 
instructional improvement, and planned change by members of the TA team (which 
informally filtered back into other program areas) . - 



Training 



MSDE-sponsored training activities related to SITTP during t|ie 1982-83 
year 'included: (1) an Assistant Superintendents* Retreat, <2) an Instruc- 
tional Leadership Conference, and (3) Follow-up Workshops on each model; Each 
of those activities led to site-specific activities at some LBAs, Training 

included' information and activities to reinforce content arid process, took 

« ■ ■ ■ ' 

into account participant needs and interests, involved local teams and outside 

' ' ■ ' ' < ' ■ ■ 

consultants as presenters, and was provided on the understanding^' that.MSDE 

would provide assistance for LEAs wishing to follow through ideas with a . 
larger, number of localj educators. The various kinds of training events 
reinforced each other, and helped to establish a common knowledge base for all 
hierarchical levels. Participant evaluation of events, the subsequent local 
requests for on-site presentations* and' assistance, and . the scope, and fidelity 
of implementation, provide strong evidence of the value to ^participants of the 
SITip-related training provided by MSDE. , . ; \ ^ 

Technical Assistance : , - . 

Technical assistance (TA) to LEAs was provided by an eight- person team 
under the leadership of the Assistant Deputy Supetintendentw TV-e team carried 
out planning and training activitifes and also. worked iri dyads ,tc provide 
model-specific assistance to local implementers, spending. a total of 263 work- 
days on SITIP. Accomplishments of the TA team included: providing leadership 
for a statewide school Ifi^yttwement program while at the same tln.e encouraging 
local ownership; maintaining communication within MSDE and among LEAs; 

developing networks and teaching/ learning opportunities for local teams to 

.. . ^ ■ ' . ' ' ' * " 

share successes arid build expertise; developing expertise among themselves . 

and applying it not only in SITIP but also in other areas; and Jnci;;.easing 

' ' ' V 

awareness of effective SITIP practices to researchers and educators outside. 



Maryland, Impact ; relating specif ically to assistance included: • increased 
trust ^nd openness in communication between LEAs and MSDE; increased effort by 
some LEAs to carry out their plans; better linkage or a clearer common know- ^ 
ledge among hierarchical levels within LEAs; increased involvement by central 
office staff in some LEAs; changes in planning, decision-making, and/or 
communication (e.g., more involvement of teachers) in. some LEAs. 

Evaluation Overview 
. This section summarizes the questions addressed by the study for the 

1982-83 school year , responsibilities:, and diata sources, measures and methods, 

- ' • ,' • . 

arid analysis and reporting. 

Questions Addressed ^ ^ 

The, study addressed four areas: impact, implementation,' dissemination, , 

and technical assistance. 

1. What is the nature and extent of impact : 

1.1 On educators, in terms of: . ^ ' ^ . - . 

1.1.1 increased knowledge - 

1.1.2 change in practice or polity . 

1.1.3 attitude to specif ic . topics and to teaching/learnirig. in 1..^^ 
general - - V 

1 .2 On* students, in terms of; " '\ ' , \ -k 
1.2.1 chang;d in achievement levels «. " . " 

^ . 1.2.2 change in behavior (e.g. ^ attendance, disruptior , homework 
completion) , 
1.2.3 change in attitude -(e.g. > locus of 'control, self-conciept, 
' . group participation, willingrte^'s to work) ' , 
• " ■ • * , ■ • 

2. What Is the nat]ur-e and extent of implementation : 

2.1 Within a local system 
.2.2 Across LEAs implementing a given model . 

3. ,What is the nature and extent of dissemination : 

3.1 Within a local system 

3 . 2 Between LEAs . ^ . ^ ^ 

3.3 Outside Maryland , - 



4, What is the nature and extenf of> "technical assistance Ry.ovlded by MSDE 
in- terms'^of : i , .. /' ' . 

^.'1 Jmplementation — - planning", training, support- . : / 

' :4. 2 I Dissemination — planning, training, knowledge base/ itiform^tion 
4,3 Evaluation'^ — planning/design, techniques, jneasures, d^ta analyisis 
and report irig *. . \ . • • , 

, ■ ■ '/ ■ V :* ■ . . . J r ' <^ • ■ . . 

Responsibilities and Data*'Sources j / 1»: , / ^ s?' ' 

. * While RBS had primary responsibility' tprVthe^SIT three' 

factors influenced the decision to Involve MSDE T^s ai;id LEA coordinators more 

. ■ ' • •-• .' . p - ♦ » 

directly 'in evaluation activities: (1) student achievement data relating to* 

'■■ . ■ ■ ' ■ , ■ ' ■. ' ' ' I ' ' • 

impact questions could best be; collected and .st^mmarized by LEAs; {ly if 

similar measures and methods Werfe'' used by all LEAs, results ^could more easily 

be compared' across the state; and (3) some LEA and MSDP staff wanted to 

•'•■/•■ ' .■ . ■ ■■ ' ■ . . \ 

improve their expertise in evaluation, by becoming more involved, 

J ■ V . ' . ■ ' ^ ■ . * - .. ' ■ ■ 

For these reasons, RBS worked with MSDE TAs to develop aiT^overall design 

and w^ittefn gxiidelines for LEA involvemeti^ The guidelines summarized the 

design, listed role group and individual responsibilities, inclucled a check- 

list planning sheet indicating mandates (e.g. , choice of various given ways t 

■ . " > • » . *■ •■ . ■ •.»■■• 

measure student' achievement) ,* and described measures and methods* RBS and 

MSDE staff reviewed the guideline^ with LEA teams and each LEA completed a 

planning sheet agreeing to a cpor\iihated evaluation effort . 

. In general, ^gS was responsible for design,* development, analysis, and 

reporting, MSE^E TAs were responsible, for/coordi:nation, distril?ution and 

" « ♦ ■ .1 ' , ' ' ■ . . * 

collection of matgrials (e.^,, questionnaires) , and assistance to LEAs in 

•following the guidelines (e.g., how fo so^e and summarize student attitude 

surveys), LEA coordinators Ck^y contacts) carried out tasks similar to 

those of TAs, but each in his/her own district, LEA eyaluators worked with 

.coordinators to collect, score, and* summarize data, particularly that 



colleqted from students. ' (Local responsibility for these tasks not only j 

itid*eased local involvement and awareness of program impact, but. also ensui^ld 

'\ ■ ■ " ' ' ■ ' ' 

that^ concerns relating td "prote'ction of human subjects" were dealt, with 

' ' ■ . .• , . - . . 

appropriately.) . 

Information—materials, interviews, survey responses—was provided ;by: 
(1) the seven MSDE TAs and the SEA' assistant deputy superintendent; (2) \Zk 
central office staff directly involved in SITIP (usually between one and three 
; people for each' of 24 LEAs) ; (3) school-based administrators (up to 10 per. 
LEA); and (4) teeachers (up to. 20 per LEA).. Also large, number?, of participants 
of ^tate-sponsored training events provided information either directly^ 
(responding to questionnaires) or indirlctly (observed by RBS) . j 

r students also completed cognitive and affective measures.* Usable data 

■ ' ■ ■ 1 ■ ' ■ . \ '. 

from cognitive mea/ures (e.g., California Achievement Test) were: summarized by 
nine LEAs (nine projects) and submitted to RBS. Usable data from affective 
measures (e.g.. Learning Environment InventoryV.weresummarized by 13 LEAs (14 
projects) and submitted to RBS. ' Two of the 13 LEAs used their own quest ion^- 
naires to measure stude«it attitudes. 

Measures and Methods of Data Collection ^ ' 

Six general methods of data collection were used: observations, inter-' 
views, questionnaires, document finaiyses, and measures of student achievement 

an ^at titude, ' i 

Observations . RBS staff :observed four SiTIP "follow-up" workshops, two 
■ ■ ■ . . , 't 

statewide conferences, and all monthly technical assistam:e (TA) meetings, and'. 



* While MSDE expected all "veteran" LEAs (19) to submit data sunmaries, 
several did not do so, for a variety of reasons. 

■ . "■ ■ * . ■" ■ . ■ . , • • ' 



also pafticipated in fall and spring site visits at four LEAs. About 400 
*pages of field notes were taken, and reports on training events were developed 
and submitted to SEA TAs within 10 days of a given event.. , - 

\ Interviews. , The SEA TAs,. LEA key' contacts, and various local educators 
were interviewed to clarify or verify perceptions or repots, ar.d (for the 
TAs) to review roles,, successes, and- challenge^, V 

Questionnaires . Three questionnaires were used: (1) the key contact 
^questfionnaire was completed by all 24 LEAs to provide "pre" and- "post" base- 
line data; (2) workshop evaluation forms were completed by participants 
(approximately 850)* of the various SITIP trainlrfg events j and (3) a general 
survey was Completed by 350 local educators (teachers, school-based adminis- 
trators, knd central office staff), wl^th items relating to prograp implementa- 
tion, impact', dissemination, and the nature -aAd extent of interactive support. 

• ■ ■ . ■ ' ■ ■ ■ ■* ' <' 

Document Analyses . All SITIP relajted-Tiiiaterials developed .by the SEA were 

reviewed as well as LEA materials used at presentations and samjples of class- 
room materials. . . ■ 

Student Assessment . While the General -.Survey included questions relating 
to perceived program impact on students in' terms of attitude, behavior, and 
achievement, data were also collected' directly from students. The SEA 
expected all 19; "veteran*) LEAs (24 ^projects) to collect data and send 
summaries to RBS. All LEAs received _guldelines>on student assefsment, 
attended workshop sessions discussing alternatives, and were encouraged to 
seek assistance if necessary from SEA .TA.S. 
Analysis and Reporting 

Local participation in data collection from studeiits was not^as High as 
was expected. Eight projects reported usablerdata based on teacher-made 
criterion referenced tests; three projects reported data from standardized 



9 11 



tests (of •which two also provided criterion data); fourteen prelects reported : 
data on studetit attitudes, sevenvosihg a brief questionnaire, and five using 
inventories (My^ Class Inventory — elementary, or Learning Environment 

Inventory — secondary) and two using their own measures. Of the 24 LEAs (29 V 

.... ■ p . ■ • ■ ' **■ . 

projects) 11 submitted no usable student data. Of those 11, ten were * 

"vefeeran" sites that had agrieed, to collect, summarize, and submit d^ta in - 

• \ .1. • ^ " . ' . • ■ . ■..■*.', 

accordance with alternatives and guidelines suggested. Reason^why (Jata 

submitted were not usable included: schedules -of data collection Sind imple- 

mentation did not "fit"; "pre" data only were submit^ted; or means reported . 

■ ; • . ' ■ ■ . ■■ ■ ' . ' ' 

summarized results of unrelated scales Reasons why data were not submitted 
at all included: - none collected because staff forgot or had more pressing 
responsibilities; lack of resources to develop' summary tables of results; or 
local staff felt that publicizing, results would cause^/inter-school competition 
or friction. Those LEAs that had no usable student data to help determine the 
"objective value" of a project, relied more on the perceptions cf staff ^ 
directly involved (the "subjective value") and, to sotoq, extent, on comments 
and comparisons made by "outsiders" such as SEA TAs, RBS $taf fy^or staff / 
involved in similar projects in othM" districts. In sbme casies staff 
perceptions were ba^ed on results of teacher made tests (weekly quizzes or V 
unit tests) or assigned grades. v * * 

Data collected were . analyzed by, project, by LEA, by model, apd by .role 
group (teachers, school administrators, etc.) . Interim. reports were submitted 
to the SEA TAs, and annual (model and project specific) reports reviewed with 
LEA teams to help in data-based decision-making for .program improvement • 



, Local Implementation and Impact 

■ ' ■ <3»^ ' » 

This section *foctises on local implementation of £tie models: Active 

' .. f^achlng (AT) , Mastery Learning vXML) , Student Team Learning (STL) , and 

Teaching Variables (TV), examiiftng the following areas: planning, scope and 

■. . ^ ■ * . . '^ ' ' 

intensity of implementation, time spent and responsibilities shared, impact, 
and participant concerns. . 
Planning / 

Objectives addressed by LEAs can be divided into four cate^;ories: 
.Student impact (objectives 1,, 2, and 9); training (objectives 3 and 4); 
teacher impact (objectives 5, 7, and 8); and curriculum alignment (objective 
6). (See Table 1.) The level of achievement varied across the four catego- 
- ries, with the highest .level,. In curriculum alignmeht followed by training, 
. V \teache^ impact, and student impkct. These results are strongly influenced by , 

6 . • . • 0 * .; • , ..... % 

the amount of time and effort that educators spent on the objectives. For 
Instancei educators have been working on curriculum alignment for Project 
V :Basic since 1979 and so have a sound foundation on which to build. Local 
-*achlev^mei|t of training objectives was influenced by the fact tl at provision . 

r >6f. information and training for SITIP began in 1980 and has been strongly 

•' ■ ■ **'.•■ '• * ' *, . • 

reinforced by both MSDE and LEA activities since that time. ^Changes in 
teachers' behavior have arisen from that training, aitd achievement of objec- 
tives in this category was strongly 'influenced by the interactive support, 
provided byvIiEA/team membeig^ The three objectives relating to improvement in 

. . students' achievement and?*attitudes can only be achieved after the other 
: categories of oDjectlyes have been accomplished. These findings reinforce, 
those of Qthet school Improvement Studies which have found that major changes 

\: . affjecting istudents take^ from three *t.o: five years to bring about^. 



iil3 



Table, 1 ' 



Status of Objectives: / All Models, June 1983 





Local Objectives " 


^ Number of Pre 
Addressing Ob- 


)jects 
ectives 


' Achievement ' , 
Status** , 




Total 


AT 


ML 


STL* 


TV 


% of 


Proiects 






N=29 


•N=7 


N=7 


N=8 . 


N=7 


•1 


T : 3,.. 


I. 


Improve student achievement ((basic' skillsj. 


26 •' 


6 


7 


6 


7, 


23 


50 27' 


2. 


. Improve student achievement (other subjects). 


,22 


3 


6 


7 


6 


45 


45 10 


3. , 


Inform local educators, about mo^el. 


... 27 
.26 


6 


■7 


7 


7 


11 


33 56 


4.' 


Train educators about mcfdel. - 


5 


7 


8 


~6 


A 


61 35 


5. . 


Improve teachers' classroom competence. 


29 


7 


7 


8 


7 


3 


59 38 


6. 


Ensure match of instruction, curriculum, 
and tests (s). 


■ 18 


1 


■ 7 


. 4 


.6 . 


0 


39 61 


7.. 


Help teachers become better organized. 


. 27 


6 

« 


7 


-.. '7 


7 


4 


66 30 


8. 


J 

Improve, time-on- task. X 


25 
I 


. 6 


'7 


5 


7 


8 


^56 36 


9. 


Improve students' involvement in learnilig 
(motivation) . ^ 


23 


6 


5 


7 


. 5 


13 


48 39 



*. prince Georfe's County did not submit data on status of objectives in June 1983. 
** Status: 1 = Hoped for , 

2 = Partly achieved 

3 = Achieved • ' • , 



/ ' • ■ • • . 

Scope and Intensity of Implementation , * 

During th^l982-83 school year, all 24;.school districts were involved in 
SITTP, 20 implementing a sin^^gle model, three implementing two models, and one 
implementing three models. (See Table 2.) 

There were six AT projects , seven ML projects, nine STL projects, six TV 
projects, .and one combined AT-TV projec*t for a total of 29 SITIP -projects 

acrogs the state. Over 986 teachers and 34,955 students in 139 elenjentaty and 

■ *i ■ 

secondary schools were involved in SITIP. Of the 139 schools involved, 65% 

were elementary, 34% were secondary, and 1% were "other" (i.e., K-12, 

* ■ ■ . ' ' ' * 

vocational-technical). Fifty-two percent of, the schools, 58% of the teachei;s, 

and 69% of the students in SITIP were using ^ the AT model, More than ,11% of 

Maryland's schools were involved in SITIP ^!aT — 6%, ML ~ 1%, STL ~ 3%, TV 

~ 1%). (See Table 3.p 

LEAs selected one of four implement^ion strategies : (1) district-wide, 
(2) pilot-district, (3) capacity building'^ and (4) lighthouse school. The 
lighthouse' school strategy was the most popular (16 projects) , fol^lowed by 
pilot district and '^capacity building each used for five project?, and 
district-wide used for three projects. • 

There was a Krelationship among the. implementation strategy' used, the 
nature and extent of central office staff involvement , and the extent; to which 
the model(s) used were perceived by central office staff to fit LEA priori- 
ties. For instance, the district-wide strategy required centra] coordination 
and coT^isiderable central office staff involvement, and was used where the 
model fit closely with a local priority. The pilot-district strategy was not 
quite as demanding and (with the exception of two LEAs) was used where the 
model fit local priorities.* The lighthouse school strategy, implemented as 



* In both cases (of exception) the model as implemented did not support local 
priorities: expansion was curtailed and central office suppont was low. 
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Scope and Intensity By County: / All Models, .lunp 1983 



CoHnty 


Topics 


: — r- 

Strategy 


1 '■ * 
//of schooli; 


Type 


//of teachers 
I 


ilo( students 


■ A 1 1 i*V''"iv . 


MI. 


LS • 


I ' 


0 


22 ' 


300 


Anne A run do I 


Ml. 


LS 


1 


H 


3 


150 


BalcLmore City 


ML 


PD 




J/M, H 


150 


3.332 


BiiJcimori' County 


^tL 
STL 


LS 
PD ■ 


3 
2 


"■' ,,'E ■ 
E» J/M 


13 ' 


325 
225 


Calvert 


STT. 
TV 


LS 
LS 


3 
3 


E, J/M 

J/M 


10 ■ 
23 


300 
540 


Caro 1 iuc 


.AT 


LS 


2 




5. 


122 . 


Car roll 


■ ML 


LS 


1 




2 


161 


Cecil ' 


■ . AT 


PD 


17 


1\ E, J/M 


^0 ^ 


2.000 ' 


Charles 


STL ^ 


LS , 


10+ 


E, J/M 


f 17* 


650+* . 


Df^rclK'sCiir ■ 


STL 


PD 


1 

• 


E 


8 


177 


Frt*d crick 


TV 


PD ■ ■ 


2 


J/M, H 


15 


600 


('arrt'L t 


A 1 






1) 


I 1 


. 44 t 


HaiJtird 


AT 


DW 


■ 3^ 


K, J/M 




1<M77 


Howard . . 




LS 


1 


. J/M 


9 


260- 


Kent 


TV 


DW 




E 


32 

XJ. 


. 676 


. lUIl L JJ,UUl(f [ y 


AT 
STL 
TV 


LS 
LS 

LS 


I 

I ■ 
1 


E 

J/M ' 
J /M • 


9 

10 
7 


170 
480 
300 




STI; 


C!J 




No data 






QuctMi Anne's 


STL 


CB 


1 


H ' . ^ 


23 


900 


J: 

Sc. Mary's j 


"at ■ 


CB 


5 . ' 


E, J/M, H 


27 


1195** - 


SoniorsL'C < 


TV 


LS 


1 


, ]L 


8 


217 


, Talbot 


y 

TV 


LS 


1 


0 




80 


Wash Ln^ton 


STL 


ytB 


U . 


E, J/M, 'h 


20 


600 


Wicomico 


AT 


/ W 


12 


E 


43 


■ 1 100 


WorcL'S ti*r 


^^. 

STL. 


LS 
CB 


k 

,1 


. t 
E 

■ E, J/M 


.16+ 


. - 75 ' 
■ 400 



*Mattlu'w Hensnn Middle School only 
*;ffTnc hidi's some duplicates 

Topics: AT=Active Teaching Strategy: LS«LlRht house school 

>n.=Mastery Learniijg • PD=Pilot district 

STL=»Student Team Learning . DW=District-wide 

TV'Teaching Variables CB«Capacity building 

Type: E" Elementary 
' ' J/H-Junlor high/mid'dlG 

H-High school 
0-Other 



- * Table 3 ^ * • » 

Sccrpe arid Intensity Summary: All Models, June^ 1983 



Models* . 


■ 

. Proiects* 


Schools'*!* 


Teachers 


Students 


#■ 


% 








% 






ACLj.ye leacmng ■ ^ 


7' • 


24 


72 

E 52 
S 20 


. 5 ?. 


572. 


58 




69 . 


• Mastery Learning 

< . . 


7 


24 


13 

E 4 
S 8 
0 1 


■ 9 


203 


21 


* 4,603 


.13 /. 


V 

Student Team 
Learning 


8 


28 


42 

E 28 
S 14 


30 


U3 


11 


3,732 


11 


Teaching Variables 


7 


. 24 


12 

E 6 
S 5 
0. 1 


-^^-^-^ 




lU 




1 


Total 


29 


100 

■ / ■ ■ 


139 

E-90 
S 47 
0 2 


100 


986 


100 


34,955— 


^io\) 



* Although Prince Geojge's County implemented STL in about 10 schools, no 
-**hard" data were available at the end of the school year. Therefore, . 
this LEA is not included in these results. 

One school is implementing two projects — AT and TV. It is counted once 
under TV. - ^ 



Scjiools: E = Elementiiry 
S = Secondary 
0 = Other 
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designed at all sites, require'd a fit between the model and the school's 
priorities (not necessarily the district's priorities) ,; and central office 
administrative support. Expansion occCrred beyofid the lighthouse site orfly 

when:- (1) there w^s impact on student achievement (objective value) ; (2\- ^, 

■ '■ ■ , « 

teachers liked the model (subjective value) ; and (3) central office staff 

' ■ . ' • / • , ■' ' ■ .>■■ -- - ■ 

provided additional support . (usually to make the necessai?y^ arrangements for 
staff in other schools to attend training).^ The capacity building strategy 
was centrally coordinated in tyo LEAs and school-based at three sites, with a 
fit between the model and LEA priori,ties at only' one of the latter. The 
greatest weakness of this strategy was that once teachers were trained, in 
most cases they had high autonomy and low interactive support (reflecting, low 
involvement of central office staff), and t^e fidelity arid frequency of 
implementation was not as great as for othet istrategied: ^' . 

. These- findings suggest that the closer a model was to existing LEA 
priorities the more likely it, was to draw central office involvement, and sub- 
sequently lead to strong and widespread classroom use. Conversely, when the 
model did not fit a district priority, it could be well implemented in a 
school where it fit that school's 'prioritiei|" but was not likely to be widely' 
.used, and its survival depended more on the individual teachers involved. 
TmplementaU^-on strategies initially s(.elected by LEAs reflected the 8b\ount of 
energy and commitment of local educators which was based on the fit — as they 
perceived it— between the model and their 'priorities. If, subsequently,: it 
became apparent that the fit was greater or smaller than at first perceived, ^ 
the strategy w^s changed. 

• . Fidelijiy relates to the extent to which teachers, implemented the models 

as designed. AT had the greatest fidelity, with 72^ of the teachers 

■* '.♦••••* ■ ' 

implementing all six components^ as compared to ML where 23% of the teachers . 

• " ' '..»•■ 

■ :■■ , ' i48 ' ■ ' • „ 



carried out all ten components, and STL where 33% of the t^eaqhers carried out 

the five required components. For'AT, no single comppaent was, addressed by 

•less than 88% of the teachers, as compared; to :ML (52%)- and STL. I76?ff.'. For TV>, : 

the majority oiE^t^je educators (62%) implemented the titge variable. Most of 

the teachers (72%) had been bbserved^by principalis and vice-principals,* and 

.45% reported strategizin^ during staff meetings^ - ' • 

Time Spent v..and Responsibilities. Shared - . 

•This section summarizes the amount of time spent on SITIP activities for ; 

all four models by each role group during the 1982-83 school ye^r. 

The average number of months* involvement by teachers for thr6e models 

/■ " ■. . ■ ■ ■ ■ . • ^ 

(AT, ML, TV) was eight months, With no one involved for less th?n five months. 

AT implementation was continuous for the specified number of months. - With thc^ 

exception of three MI^ sites, teachers did" not use IJL and STL cortinuously 

during' those months.^ TV teachers were usually^observed at the beginning and 

end of the time, applying improvement strategies in the interim, if appro- 

priate. During the period that teachers were. directly involved, the average 

time spent during a given week ranged froitf 21% (STL) to 39% (AT). For three 

models (AT, ML, STL) elementary teachers spent less time (15% tc 23%) than did 

Secondary teachers (23% to 51%). This reflected the fact t:ti^t elementary' 

teachers used a model for only one or two curricular subjects,_vhile any 

secondary teachers involved used the model for his/her subject area specialty 

with a relatively large number of classes. For TV,' secondary teachers '.^speht' 

19% of their time and elementary teachers spent 33% of t he i^-fejjre^^t^ 

subjects for which "time" observations were conducted or the "content" . ' 

variable addressed. This reflected the higher credibility which the TV data-. 

base had among elementary teachers. 



. In order of investment of classroom time, models were: AT, ML, TV, STL. 
Investment varied from one site to another, influenced strongly by administra- 
tiye. decisions and the amount of development work completed in the first year. 
, Factors working against, high investment of classroom time included: (1) need 
for materials (STL, ML); (2) need for preparation time (ML, STLV; (3) pressur-e 
to cover the curriculum in a given ainount of time (Ml., STL) ; (4) relative - . 
suitability of a model to the curriculum (STL); (5) relative suitability of a 
modfel ,to a grade level (TV, secondary) ; and (6) negative experiences in early 
implementation which were not totally resolved ^)y local administrators .(some 
sites tor AT and ML, perceived most strongly. for TV) . Factors facilitating 
high investment included: (1) ffvailal^illty of "materials (ML, STL); (2) low 
complexity of the model XAT) ; (3) suitability of the model to a curriculum and 
grade (AT); (4) successful application experienced by teachers early in the 
project (AT, STL); and (5) successful^ application facilitated by local 
administrative support (all models in some sites, but perceived most strongly 
for STL). ' ' ' ■ ; \ 

The average amount of time invested by central office staff and school- 
based administrators ranged from nine or ten days for AT and STI, to 23 days 
rfor ML. Individual administrators spent as few as two days on SITIP to an 
ali^ost full-time commitment. With the exception of TV, central office fftaff 
spent almost twice as much time as school-based administrators. In all cases, 
combined time of administrators was invested least in materials identification 
and/or development.. For three models (ATi ML, STL) most combined time was 
spent on supporting school implementation and administration* the/other three 
areas of activity — inservice, dissemination, evaluation — took relativelv '' 
little time. Since appropriate materials were essential for ML and STL, and 
since administrators invested so little in this activity, classrDom use was * 



reduced unless teachers already had materials or were given release time for 
development. Given teachers* concerns and needs relating to TV, greater : 
inves'tmeht . in support (rather than administration) was desirable at some 
•sites., ■ . • . : '. 

. Interactive 'support included both logistical and affective support: 
information exchange; training (both traditional inservice and one-on-one 
c'baching) ; provision of materials and other resources; arrangements for 
teacher release time; assistance in development of quality materials, tests, 
'tecord-keeping systems, etc.; acknowledgement and publication of success; and 
supportive use of feedback to encourage improvement. Support was rated for 

all role groups by the three local role groups (on a- five point scale, l=very 

. ' .1, 

poor, 5=excellent) , and results" are presjented for all four model s in Table 4. 
While ratings of developers' support are relatively unimportant at this stage 

i . ■ •. ■ ' ^ 

of implementation, the somewhat low ratings for central office support (below 

^ ■ ■ ' 

average — 2.98 • — for TV, to 3.82 for STL) are of concer-n where projects are 

not school-based, and where the LEA expects SITIP implementation beyond a 

single school. ' . 

- • , • ■ 

Impact . . . ,^ ' , 

,, » ■ 

This section discusses impact for all models on school syst^^j^, central 
office staff, schools, school administrators, teachers, and students. 

As can be seen in Table 5, the most common impact at the district level 

• • ■; ' ■ ■ • ^ — -^5= 

was the commitment and sharing among educators (reported for AT and ML) which 
was encouraged'* by the SITIP design. Also, for Jtwo models, policie^^ere put, 
into practice to facilitate implementation and encourage institutionalization. 
, Knowledge of a new teaching or observation strategy was noted by central 
office staff for all models, plus acknowledgement of AT*s influence on 
improving organization for instruction. The strongest area of impact at the 



„4 



...... Tabled ' 

» 

Perceptions of Support Received: All Models, 1982-83 ■ 



^-^.s^.^ . ^ Support Groups 
Models/Respondents^^'*'-**-,,. 




['Teachers 


School ' 
Administrators ' 


Central 
Office Staff 


MSDE 


Developers : 


■ , 1 ," ' ' ' ' 

Active, Teaching . 


112 


^ -3.79, 


3.83 


' 3.79 , 


.3.61 


3.:24.-- 


■ 'Mastery Learning 


■ 75 , 


V..11. 


/'3.95 


^- . k81 


'"3.64 ■ 


' 3.21 


Student Teani' Iiearning 


86 


4.ir ., 


4.12 


3.82 


,3.94 


• 1.72 . 


Teaching Variables 

■A 

Totals 


61 ' 
334 . 


3.74' 
3.94 


3.90 


. 2.98 ■ 
3.65 , 


3.29 , 

.3.90 
\ 


2.90 i 
„'■ 3i29" { 



\ 



Mean ratings range from a low of l.OJ (very poof) to a high of 5.00 (excellent). 
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Impact nf Tmplemcntation on Administration 
Schools, and Districts: All Models, 1982-83* 

















Role Croup 


s and Models 












Impact** ■ * . • 


School Systems • 


Central Office Staff , 


Sc 


loots 


School Administrators 




AT 


ML 


STI, 


TV 


■ AT 


ML 


STL 




AT 


ML 


STI, 


TV 


AT 


Ml,, 


STI. 


TV 


Knowledge of a new teaching strategy 
Knowledge of time-on-task. 
Knowledge of* effective observation/ 

supervision method/criteria 
Knowledge of '.learning (theory, practice) 






2 

;> 




_ ,. 

2 


3 
2 


3 


1' 

5,:' 


• 




3 


4, 


3 
3 


5 


3 

V 


2 
3 


■ ■ , ■ " 
Belief Injraditional teaching 

Commitment/sharing among educators 

Continuity/consistency across classes . 

Interest/enthusiasm of students/teachers 

(e,g, , in subject area) 
Better management, organization or ' ' ' 

instruction ^ . ' 
Appreciation for teachers', recognition . 

.of success 
Support- fe.g.; arranging common T)lanning 

tim) for teachers 
Closer monitoring of teaching 


' 2 






> 

♦ ■ 


2 


k 






2 


5- 
■ L 


2 

■ 4 


3 
\ 

3/ 


2 

2 

'2. 


2 






. • ■ ." '■. 

Policy to releasjB teachers to train , 

others or cbordin^te^rqject 
Policy to Implement for a given subjtict 
' of grades / '^ * ' 


f ■ 


\ 








' V ■ 








: ~"'V 















Rep'^rted In number of LKAs: . " - . * *- *.. , 

•Active Teaching N'?,, Mastery Learriihg'Nr7, student Team Learning Teaching VaTiflbles ^^=6, 



Impact areas reported only *whnn stated hv *tvo or morc IIAs for a given model. 



school level was . sharing among educators, with continuity and consl'stency ; 
across, classes. Interest, Appreciation, and support were valued, as was' ■ ■ 
closer manitoring and better organization for instruction. Principals and 
other school-baseid administrators for all models valued new teaching or 
observation strategies, gained an appreciation for teachers* capability (AT, 
ML), and strengthened their belief in tra<iiti^)nal teaching (AT). . The impact 
of involvement in SITIP, in: terms of teachers * understanding the models and 
improving. their teaching ability, is^summarized in Table 6, with the percent 
of teachers noted for each area of impa^ for each model. The relatively low 
perceived impact of TV may have b^een influenced by the amount "of training, by 
the lumber of teachers who were found to have satisfactory, engagement rates - 
(time-on-task) and therefore saw no need to change, and/or by the interactions 
between observers and teachers. ^ . , 

Table 6' " '\ ' ■ • 

Percent of Teachers Impacted by Involvement: All Models, LSSZ-^SS 



Models 

Impact 


AT 


ML" 


STL 


TV 


. - • 

teachers understandin|^|odel 


72 


73 ' 


80 


64 


teachers improving teaching 
ability 


66 


64 


50 


' 28 • 


teachers seeing no change in 
teaching ability - 


13 


16 


2.7 


36 



Each of the, three local role groups rated impact on teathers in terms of 
enjoyment, increased knowledge, and increased skills (on a f-iye point scale- ^ 
where 5.00 = strongly a!gree) . Responses are summarized in Table .7. Mean 



Instructional Impact as Perceived by 
Survey Respondents: , All Models, 1.982.-83 



( ' ■• • ■ . 

■ ' ■ ■ ■ ' '•. * ■ • . ' ' ' " 


■ • ■ ■ " " • , ' ■ ' ■ • ' ' 

Models .. ; ; 


,*'.,',"'.' ■ 1 ' 

Impaict on Instruction . . ^ N= 


AT , •„ 
122, 


. ML 
. ,"76 > 


: STL 

.89; . 


TV; 
•63 


Total : 
-350 


Instructional Value , : 




• 




^ 

■ ^. . 




. Works in classroom, ^ 
Is worth the work it takes, ^ ' 
Is a worthwhile teaching approach. ^ 


4.29 

;4.38 


4.35 . 

. '3',85' ;■ 

.4.28. . 


i4.42- ^ 
;4.18 , 

4.43 .;' 


:3.73 • 

;3.79 


4.35 , 

4.07 

4.26 


Impact on. Teachers' 








I-- 

* . » 


■ . > _ 


Teachers len joy it, ' . . 
Teachers ha^e increased knowledge. 

Teachers have increased skills. 

1 ■ . ' . .. . . ' ■ ■ * . ' 


4.02 
4.08 
.4.05 


'■'■'■■3.7*? , j 
. 4.08 ' 
4.08 • 


4.17 
4.14 
4.06 


.3.56 
'1.60 
3.51 


■,1.92 ^ 
. 4;.01 : 
3.96 • 


Impact on Students . ' : \ 












.. Students enjoy it , * / . , ■ 
■Students are less disruptive. 
Students' achievement has increased. / 
*Students are learning more^ 
Students' general behavior is better. 


3.88 
3.88 
3.59 
3.61 
3.73 


. 4.09 ' 
3.16' 

. 3.76 . 
3.67 • 
3.09, 


4.37 
3.65 ■ 
3.76» 
3.60' 


1.52 

3.11 

3.08 

2.81. 

3.08 


3.99 
3.62 
3.58 
3.48 

3.''4'3,>^ 


Time ' ^[ ^ 












Teachers spend more time preparing students. 
... , Teachers cover curriculum in less time. ' v 


3.09 
3.13 


■ 'A. 01 
2j>51 


3.93 
2.56 


2.69 
2.79- 

1 


3.43 
. 2.79 



Mean ratings range from 1.00(strongly disagree) to 5.00 (strongly agree). 



■ ■ • ■ * ■ . . . . ^ .. ■ ,.,«...■ 

M=Active Teaching, ■ML=Maste,ry Learning; S1<'l,=Student Team Learning;. TV=Teaching Variables 

O 
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ratiings>iii aiy: cases indicated that impact on tgaphets in ^11 areas .dfd, occtir 
to'feWe extent. with greatest certainty among-iocal educators for, SXL and: , . 

• . , . .'• ■ ' • .■ ^ ■■ " , . : ' ■ . . • ■ 

least-'-fjar TVvT -,..'? - ' ■-. . ' '. ■ ^ . ' • ^V- '.; ■ , •,. 

.^riioie s^BcUlc kinds Of, impact on teachers i- iii terms of increased know- • 
ledge and skills and streng.£hened attitudes, are summarized in .Table 8. For 
each .fiw of Wact f or each model, the number ot LF:A-S;Wl^ere ' that, impact was 
^found^ i^ ' presented-. ^Sin:ce most LEAs hoped that . teachers would improve skills 
relating tp, instruction impact in 1;l>at category is particularly important. 
Since each mbdei emphasizes, p^rticula^ activitie?V coin>ai;isons are not always 
relevant^Vtfiwever,- the first thiree" skill areas listWd are addressed bj? all 
^6^^x models, and- results indicate that a large pum1je;i,of LEAs found that .. - 
'teachers made, improvements in ,.t^aching/obseryiftg-, classroom management , . and .-. , 
^ssessing-ind addressing student' needs. ^ For t^ree. models AT, ML, 6T^ 
results indicate that. in 50% or more of the LEAs, impact bn teachers reflected 
/the. objectives or ; clkitas o^:tli^.mbdel(sy" implement^^ For .TO,, appropriate . ^^ 

- imp^ctVwas weaker Ce.g:;y knowledge of time-^n-task in 3^% bf LEAs, skill' in 

; 'effictiy^-Aise of.l:iine|h 33% of. LEA^)r V?^idh, m^^^ fact that , 

S6.B%; of:;'teachferi did ,rtpt .need to -make^ohaiigfes . tp Imp^ time-on-task, or may 

- tiaY4-::be^i'iri^id^ foi^'Wrcrvemei^t^j^ of 

^ iichfets ^f|i-iii^^ in - team strategizlx^>';;.ia g^era^^-^^ on teachers 
w^ pbkitive and clearly r^i^t^: t^vife-TO^ 

: Impact on students a,s perceiv^'fe 
7 and. 9. For eacji kind of iiSpact £"qrt^.^a^ the number,; pj^:T.EAs .wl^r e^^^:. - v 

that impact;. was found is presented. " Across all models, the strbii^est areas of 
impact perceived by local educators were: improved -student attitudes toward 
learning and scKbol and abput their ability tp learn; Increased student 
achievement as indicated by test sccres, and mastery and retentUcn pf facts 



■ ' ' Table 8 , ' 
'impact of Implementation on Teachers: All Models,. 1982-83 



.Impact: - teachers have... '•■ 



N* = 



Models 



AT 
7 



ML 
7 



STL 
8 



Increased . knowledge • , i ' ' 

-of components or procedures of effective teaching 
' !-of timeron-task 
.■•'-of curriculum alUgnmenf and proiiram, ■, ^ 

-of research and learning theotv 
V -about teaching and learning thi^ough staff 
•, ;,(fevelopraent/ observe tion : f 

■ ^Improved skills ' r 

-in a new teaching/observation technique 
, -in classroom raanagement/organization/planning 
: -ill assessing ancJ addressing student needs 
-in. specific .components of effective teaching 
-in effective use of time 
-in use of peer tutoring 
-in working with students (e.g., motivation) 
-in curriculum development 
-in instruction 

. Strengthened attitudes/perceptions 

', -about teaching ' , 
'•, -of teachers' confidence or self-image 
, .",-ro'.f the value of traditional teaching - 

-of the value of specific components of effective 

, ■ teaching 

. -,that the larger group must be emphasized . 
;;V'-af what' students can accomplish ' ; 

. ". Tof how well students- can work together 
. . ."^bf the Importance of keeping students on task ^ 
.■■ ,-that teachers.raust teach- every day 



^5 
6 
6 
3 

7. 



3 
.3 
2 
1 



4 
3 



7 
7 
3 
A 



6 
2 



/ 
6 



* N'is the number of -LEAs Implementing a given model. , . ■ ' 

,AT=.Active 'Teaching; >IL=Mastery Learning; STL=Student Team Learning; TV=Teaching Variable! 



Table 9 

Tmpact of TmplementnCion on Students: All Models, 1982-83 



Impact on Students 



AT 

7 



Mod els 
ML 
7 



STL 

9 



Improved attitudes or awareness 

-about their learning ability 

-about their learning responsibilities ^ 

-about learning/ school 

-of their strengths and weaknesses 

-about tests * * 

-about classroom behavlor/tlme-on-task 

-of teacher interest 

-of value of being, organized ^ 

Increased achievement 

-in test scores 

-in- grades . 
-in general 

-in mastery/retention of facts and skills - 
-in problem solving and conceptual understanding 
-especially £or lower achievers 

Benefitted from better instruction which provides 

-a structured, consistent format 
^ -a clear understanding of teacher expectations^ 
-a greater variety of activities 
-effective learning activities 
-a inoriB complete instructional program 
-better usie^of time/more materials covered 
-opportunity for independent work 
-opportunity to^relearn (af.ter **no fault" test), 
-opportunity to advance 
-special benefits for slower students 
-more Individualized instruction 
-peer tutoring/working in groups 
-large group instruction 
-more organization ' 
-more attention to academic content 
-fewer gaps in skill development 
-competition ^ '^P;- 

-less pressure 
-recognition of suc<Jess ' 



3 
2. 
7 



// 



n 



4 
2 



4' 
4 



4 
1 



2 



-I- 



3 
8 



4 
4' 
2 
3 



N is the number of LEAs implementing a given model. 



AT=Active"^eaching; ML=Maste^^ Learning; STL=Student Team Learning; TV=Teaching. 
Variables. ' • <igi^;t " ■ 



29 



26. 



and skills; and benefits derived from instruction in a structured, consistent 
format with a clear understanding of teacher expectations. In general, • 
educators offered statements indicating that ML, AT, and S1^. had somewhat more 

impact on students than TV. 

Student attitude data were summarized by proje^-ts uslnH either a brief ' 
questionrtaire or a longer iijventory (Learning Environment Invent oig^ for grades 
5-12, My Class Inventory for younger students).* The questionnaire was used 
for three models by seven LEAs (Allegany, Baltimore County, Carrpll. Cecil, 
Dorchester, St. Mary's, Worcester). Results were positive for all items at 
all sites. Students, knew the difference between SITIP and regular instruc- . 
tlon: They found the lessons relatively easy, enjoyed and understood them, 
considered that in comparison to regular, lessons they were. better, and • 
students learned more and got better grades. Overall, this last criterion ■ 
(better grades) and the first (ease of lessons) drew the least certain 
responses' from students, with the total mean on those items pulled down by 
responses from older students (grades 4-12) - possibly because they are mor 
discriminating than younger students., In general, meart scores for-STL were 
higher than for the other two models. . 

Results of the inventories (Anne Arundel, Baltimore City, Caroline, 
Charles, Worcester) indicate that the means for each project and across the 
■ three projects were better ttian national norms for secondary students on four 



* The T.Parn^n g Environment Inventory (LEI) measures 15 dimensions, ^eight of 
which were relevant for assessing impact on student attitudes. Four dimen- 
sions are included in the My- Class Inventory (MCI). Each is defined: 
Competitiveness—Students compete to see who can do the best work; Satisfac- 
"t ion— Students enjoy their class work; Difficulty— The wotk of the class is 
difficult; Friction— There. are tensions, among certain groups of students 
that tend to interfere with class activities; Disorganization— The class is 
disorganized; Apathy— Failure of the class would mean little f.o individual 
members; Favoritism— Certain students are favored more than the rest; 
Environment— The books and equipment students need or want are easily v 
available to them in the classroom. ' • - ■ ' 



dimensions: satisfaction, difficulty, apathy, and environment. There was 
room for improvement in relation to friction for all sites, for favoritism at 
all secondary sites, and disorganization at two secondary sites^ There were" 
no significant differences between models, Tegardlesg of :the^ fact\hat STL is 
designed to reduce friction and avoid favoritism. 

Cognitive achievement data from standardized mathematics tests were 
reported by four projects -- one in AT and three in ML. In all cases, gains 
were greater than normally expected, -with most signifi^cant improvement founa 
for low or middle achieving students. Eight projects reported data based on 
teacheir-made criterion-referenced tests (AT=2, ML=4\ STL=2) . In most^asesf 
SITIP stude'Ats did better than students in "regular" classes, with gains made 
most consistently by below average students.: Data supported claims for ML 
tha^t at least 80% df the students achieved ipastery (established at 80% or more 
of the course objectives mastered). 

^ These results support developers' claims for AT, ML, and STL. However, 
direct cause-and-ef feet. conclusions should be made with caution, attending to 
the nature and extent of implementation relating to a given set of results. 
Participant Concerns and Recommendations* 

Concerns were reported by participants of all projects, and were 
categorized as being related to the model(s) or to the general process of 
implementation. (See Table 10. ) , Most model-specif ic concerns relied to 
management ~ the need for time and materials' for effective implementation. 

~~ 1. . . " 

* In the 1982 report, concerns were analyzed using the Stages of Concern (SoC 
developed-^y the Center for Teacher Effectiveness dt the University of 
Texas). In general terms, that same framework, is used here. Stages are 
roughly developmental (Awareness, Information, Personal, Management, 
Consequences, Collaboration, Refocusing) as an individual or group learns 
about an innovation, uses it, and fits it into existing activities. 
Concerns in earlier stages need to be satisfactorily addressed before 
participants can be expected to move to another phase of activity. 



Table 10 

Participant Concerns: '^All Models, 1982-83 



Concerns 



Model Concerna 



Time 



Materials • 
Students 

Discipline 
CurrlculunT 

Teachers 
Assessment — 
Design - 



allocations too rigid 

requires too much -record-keeping/ 

paperwork _ ,L. 

requires too much stunenj; testing 

requires too much la genAal 

requires too much preparation/ scoring 

need enrichment activities 
need materials that fit LEA curriculum 

holds back talented Ss . 
remedial Ss go off task 
weak Ss depend on strong Ss ■ 
• absentees hard, to handle 
grouping- Is difficult 

' less teacher control, more ;nolse 

■ does not fit all subjects/grades . 

■ coverage Is reduced n , 

- creatlvl-fy^ Is Inhibited 

- observation creates fear, pressure 

- model more useful for new teathers 

achievement Is dlff Icult < to measure 
point system (bumping) Is not popular 
checking should not be done by Ss 

lack of research base 
complex, difficult to Implement 
coding categories are Judgmental 



Implementation Concerns 



Insufficient time 

Insufficient central office support^ ^ 

Poor coordination (model, materials, management) . 

People and resources not used to meet project needs 

Inflexible budget process' 

Unclear evaluation guidelines 

No monitoring of observers 

Leaving class to teacher substitutes 

Poor communication — teachers don't know why model 

Is used ' 
Poor attitude/teacher apathy 
Teachers not engaging students It) learning 



Models 



AT 
N-7 



^ I 



ML 
N-7 



STL. 
N-8 



N=number of LEAs Implementing a given model. 

AT»Actlve Teaching; ML-Mastery Learning; STL-Student . Team .Learning; TV«Teachlnp 
Variables. . 
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Many concerns ^related to consequences — the impact pn particular kinds of 
students, on curriculum, and on discipline and the assessment of that impact.^ 
Teacher concerns are also consequential, with some person'al overtones. 
Concerns about the design and .some of the assessment concerns were related to . 
ref ocusing — a dissatisfaction with the model as implemented and a desire to 
do something different. General, implementation concerns were reported for 
three models (there were none for STL). Xll of them related to management, 
with some personal or consequential overtones. These results are what might 
be expected given the age(s) of the projects. . ' - , 

Recommendations were made.; by participants of all projects, and were 
categorized into six general areas: learning, teachers, classroom use, imple- 
mentation process,- interactive support, and expansion/revision.' (See Table 
11.) Learning recommendations related to the SoC "information" stage, and 
reflected a cycling of sophistication and appreciation for on-going training 
and assistance: participants have learned and want to continue learning ~ 
sometimes in a particular way or in a particular area, of expertise. Recommen- 
dations for teachers related to the "personal" SoC stage and indicate that in 
some cases there is fear, resentment, or confusiop that needs to be overcame 
(ML, STL, TV). Classroom use and implementation process recommendations '..^ 
related to two levels of management, and indicate that local implementers have' 
become sufficiently familiar with the models to identify (and want to over- 
come) barriers to successful use. The AT recommendation for situkional 
adaptation suggests a need to clarify understanding of the model (how it is 
explained, and how it is implemented). There were fewer management recommen- 
dations for STL than for otlher models,, which is somewhat surprising, given the 
number' of concerns about time and students. Recommendations classified as 
"management" were influenced by interest in consequences, jrhe "collaboration" 



Table 11 

Participant Recommendations: All Models, 1982-83 



Recommendations 



Models 



AX 
N-7 



ML 
N-7 



Learning 



Teachers 



Classroom 
Use 



Implementation 
Process 



Interactive 
Support 



Expansion 
Revision 



provide training and follov-up assistance 
'ptovlde research updates on school Improvement, 

teacher effectiveness* 
provide research results bej^ore Implementation 
encourage teachers to increase knowledge & skills 
provide more specific Instructional Improvement 

Ideas " 
allocate resources for classroom observation 

reduce burden on Ts . , 

have only voluntary pattlclpa'tlon (It's not for 

every T) 
help Ts see value of model 
compensate Ts for after school activities 
* 

allow situational adaption 
maintain fidelity (and monitor) 
allocate/adjust use of time 
provide materials 
sequence units more carefully'- 
have ability grouping/smaller classes 
develop record-keeping system (computerized) 
develop strategies to deal with absentees 



\ 



allocate time (development, paperwork, 

preparation 
assign new leadership 

do not add model on top of LEA priority 
do not begin In first few days of semester 
use earlier In the year 
evaluate effectiveness , 

increase funding 

Increase central office support V'r 
Increase MSDE/central office cooperation to 
help Ts solve problems /■ 
^ Increase MSDE assistance -.V; 
encourage more networking among and within LEAs 
provide or develop materials . 
Involve more Ts In curriculum development 

drop the program 

Increase involvement schools/grades/subjects 

try another model 

use every day ' ■ 

use for the. full year 

use another part/technique of model 



N-number of T.EAs implementing a given model,. 

AT»Active Teaching; ML-Mastery Learning; STL-Student Team Learning; TV-Teaching Variables. 
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stage related to what others are doing - interactive support - and most 
recommendations, in this group indicate that school-based staff are not ready •■ 
to take full responsibility for implementation (and perhaps should not be 
expected to do so). Recommendations about expansion or revision related to 
the "refocusing" stage, and mostly indicate that local educators value tKe 
models enough to want expansion (although opinions are divided for TV between 
expansion and termination and reflect concerns about the design and the way , 

some teachers in some LEAs react to it) . 

If a project is to succeed, concerns and recommendations should be 

add^ressed by MS1J| TAs and LEA teams. For AT, the most critical issue is local 
perceptions of th^ fit of the model to specific grades, subjects, or students 
,(as grouped) . For ML, the most ^critical issue is cost-effectiveness 'in terms 
of time allocated for unit and test development, and the subsequent record- 
keeping, in relation to the perceived Value of the model. For STL , ^the^ most 
critical issue is cost-effectiveness in terms of teachers' investment in 
relation to impact (including discipline) on various kinds of students. For 
TV. the most critical issue is the perceptions— fear, apathy, resentment 
(pritnarily of .teachers)— about l6car implementation decisions and about the 
model design. While those issues suggest negative impact in some sites, it 
should be noted that- they are not peryasive and do not out;-weigh the positive 
impacts reported earlier. 

Conclusions • 

While processes of implementation based on the research on planned ch^ge 
• wer^ecommended for all models in all LEAs. and TAs encouraged lockl educa- 
tors to attend to such principles ks participatory- decision-making, two-way " 
communication,, training and support, and appropriate investment of time and 
energy, those processes of ' implementation arid principles were not always 



applied.* When they were applied, implementation went sufficiently smoothly . 
for energy to move gradually from establishing structures, relationships, and 

-expectations toward actual classroom use,; When there were arbitrary adminis- 

■» . ' ■ . ■ • ' 

trative decisions, top-down or incomplete communication, low support by 

I * ■ . . 

• ' . * ■ . y 

central office staff, and insufficient time allocated for materials develop- 
ment or group planning by teachers, implementation, problems occurred. 

At the local level, the^jj^rinciples or practices were generally referred 
to as interactive support, and, depending. on the nature and extent to which 
they were applied, had positive impa'ct or created barriers to success, (See 
Table 12.) 

Impact was made on student achievement by three models. (AT, ML, STL), 
with the strongest evidence of success in mathematics and reading/ language 
arts for AT and ML, Positive results were moJt apparent when either of those 
models was used consistently over. a period of time for a given subject and 
grade, • ' ■ 

Impact was made on student attitudes. to some extent for all models. Data 
summarized by 12 projects (AT, ML, STL) indicated that SITIP students enj-oyed 
the lessons, 'did not find them difficult, and wanted to succeed. Friction 
among, students, and their perception of favoritism and disorganization needed 
to be addressed at some sit ek. While teachers believed that for STL students 
self-esteem and willingness to work with others increased, student^ data for 
STL indicated no differences for that model In comparison to AT or ML, 



* In some cases, the responsibility for the low l6vel of application was 
shared with the assigned TAj In other cases, the TA's efforts were 
- disregarded' by local staff. 



Table 12 

. V, 

Barriers and Facilitators to Successful Impleme'ntation 



Barriers ^ 

... . 1 


Facilitators 


Heavy reliance on training (ML, STL) 
Insufficient resources for trflln-tni? ^<^TT^ 
No foUow-up assistance (STL) 


Training and assistance responsive to Ts' 
exprefisea neeas \ax^ ml, blL, iV) 


Rationed resources, btoad dpyplnnmpnt ^MT ^ 


1 time 0 sKiii to aeveiop materials (AT, ML, 
STL) , . ' 
Resources allocated for development 


T adapt, model (STL) • 

T perceive no credibility of model (TV) 


Fidelity understood, advocated, & .acknowledged 
by SA & CO (AT, ML, STL> '\ ' ; , 


CO maintain administrative control, but expect 

work to be done bv Qrhnni cfaff u'l^hnl1^ Vmfl/l-InD 

ownership (AT, ML) 


CO demonstrate interest, in project success (AT, 
HLj.biL, ivj atid acknowledge T errorts (AT, 
STL) CO act to overcome problems ' (AT) 


Plans overly ambitious (STL) 

Plans not followed by project leaders (STL) 

Purpose not clarified, mutually understood (TV) 


Shared planning, purpose settings decision- 
:makirig'(ML, TV) . ; .:: . ":* ; 
Networking encouraged (ML, TV) 
SA fit model to school priority (TV) • 


Ts percelv? their efforts are devalued (ML) 


•u 

Ts believe their opinions and, efforts count 
(ML,TV) ' ■ 

Ts believe project is designed for improvement 

,(TV) 

■ Ts value tecommendati6hS ;qfc^ 
SA emphasizes professional' development (TV) 


Single energlzer with low influence (AT) ' > 
Conflicting messages (CO, SA, soipe MSDE) (ML, 
STL) ■ ' ■ 


' . ^ .'^ , ■ ,, ■ ' '„ ' • . 

. ' • . '■ • . , ■ . ■ ^ f ■ < 
*■* , 



AT=Active Teaching; ML^Masteiry Learning; STL=Stud6nt Team Learning] TV»Teaching Variables. 
'CO'central office staff; SA=sch6ol administratorsXj^teachersi'Qrt - ■ ■\ 



Impact was made on teachers' knowledge for all models through training. 

•Skills In a new teaching/ observation* technique Increased through classroom 

practice and coaching. Positive attitudes about teaching were strengthened as 

teachers experienced success. 

Impact was made on a school (the faculty, and how Instructional matters 
*> •• ■ . - - ' ■ 

were dealt with) through commitment and sharlnjg among teachers (ML, STL, TV), 

and provision. of support^ (ML) and recognition of success (STL), by school 

aldmlnistrat or s (usually the principal). Staff interest in teaching/ learning 

increased (ATi STL); there was more continuity across classes (AT); betters 

management of instruction (TV); and closer monitoring; of teaching (AT),,. 

^ ' , , ' . ' ' ' - ■ .. ■ • 

'■ • . - ' ■ ■ ' 'V'-- ' - ...... ,. . ■ • • ' •". A 

Impact was made on school administrators' knowledge^ for all models 
through training, and they improved instructional management t^AT) , ! 
Strengthened their belief in traditional teaching (AT) , and more appre- 

ciative of teachers' capability' (AT^'ML) as Implementation occurred in their 
'schools.-^.. ■■ " •■ 't' ■ ■ *"'■ 

Impact ^was made -on central office staff's knowledge fj3r all models • / ' 
through training, and, for AT, tHey improved insiti^ictiorta^ mainageraent as they 
became ^involved in implemehtiation, .'^'-^ ■; 

^ . r^t the sys^tj^m level*^^^ was knowledge gain (Sli) ,"' cross-hierarchical . 

sharing 'and commitment (AT, ML) ,. and policies enacted to release J:eachers to^^A^^; 
train others or coordinate ^ctiyitles,.:(fflj)-./^^^^^ to im|)lement the model ^ ^ ■■ 
district-widfe for a 'giv^^.tt'sulij^^ . 

As stated earlier, the implementation strategy u6ed inf luenc^fc^ ^PP 

(with capaclty-bulldlhg being the least effective). Another str.dng;'!^^^ 
was the' relationship between a model and localj priorities (as perc^lvedv^by 



local educators). Probably the strongest influence on, successful impleraenta- . 
tion was^nteractive support: while teachers can and do teach alone in their 
own classrooms, they: do much better when their efforts and ^,uc.c.e«se,s . are 
acknowledged and they are part of a crosfl-hierarchica]. team.working toward 

■ * .. ." . y' ♦ •- ' 

instructional improvement which benefits studet^^^^^ 

. t", . . Summary . and ■Conclusions . - . 

Application of the research on planned change facilitated implementation 
of models of instructional improvement. The SITIP design encouraged collabo- 
ration, increased communication using a comroori knowledge l^ase about-i3^^ 



dlassropm elfectiveness, and helped^LEAs estab.,J.lsh cross-hierarchi'ial teams 

■■'/■> ' " 

with ''the purpose of improving instruction, ^^jjnl^ss the principles of planned 
c'liange "were applied, the model adopted had little chl^npp;b| success. 

The models themselves were perceived by ^^cal eijucj^tors as having both 
subjective and objective value: Teachers' positive opinions had just . as .much^^ 
" influence as s^aiiiji^d^e^^ in determit^ing' pto^ram maintenance or 

expansi6n. Te^&eisf'-^^^gktd^S^^^ concerns;^dia little influence it^ ■ 

determining mainJienknte^ ior^jeXp^^^^ influence. -ihepiSelative impact of,- 

.4:i)e project. '■;;V'-W:_:^-^^.5^r^^^^^^^^^^ . , ' ,. ; 

\ Active' Tfe^ching and Mastery -Learning, when implemented-'with fidelity for 
a complete cour'^eV had a-^positive impact on student achieyem^f t , and helped. . 
teachers to oig^ize 'instiruction effectively.' The models were valueid inore h^ 
teachers •wh6nViiiedi' f di^; :fitruct.ured academic curricula than . for more open-ended 
subject areaf..; Ma's.t^^^ Learning required considerable gJdministrative support. 
Both models were^ior^s^c administrators acknowledge teachers' 



' . ^ Student ' T^am Leaigiing was popular iyith studehtS/a^^ teachers and had a 
positive, fmpact on achievement in some cases. Howeve'Ti;^ it was not used 
consistently, and so c;|use-and-eiEf ect claims* cannot Maintenance 
and expansion usually occurred when teachers saw th and 
appropriNate mkteri^ls were ^■::C':Ji'^^:^:z 

Teaching Variables .Was used as 'a vprofesfeiorial deyeloptae^ (arid', 
^as. then more likely to be valued by participants) , Qr^^ 
sion process; Xa^d was .t.heri mpre likely ^^^^^^^^^ 

t-eachers) . Littl e' evidence was prov4.ded to indicate^ impiact . on rStii'dlB^^ 

-'' afchievement, but there were some repdttis of teachers improving their manage- 

V ' ' ■ ' •:. ■ ■ " '11 'l-'r*' ■ • . . " 

ment of instruction, ■ ^ ^ :• ' 

: -Key staff in>aill LEAs, in 11%. ofl Maryland's schools, irt^olleges of . * 

education, 'an&'''ktl MSDE increased theit understanding of recent research, on 

platin,e/i change and school arid classro9m effectiveness.' Nearly 1^ teachers 

modified their instructional techniques, ; 'and: most of ^ them believed -i^that the 

results are worthwhile ^ i The g^feneral attitude of all role groups involved in 

, - .. .lis.*- . '':M /, : 

^SITIP was positiye>'j^!^it:h^ appreciation ^^f opportunities for professional.^ 

•; y^growjili, and for thei benefits to student}s receiving; instruction, 



During the 198^8)|\'sch6 year, lo^al imp lemeri^^SBh will continue to be 
supported by MSDE, wit^ attention to part|,ciig^;?^ 



'and to the resul 




ed here.. SllalP advocates hope that JLEAs will make 
cisions' ~ either to terminate or tp institutionalize 



;,fer with local comitment to .tfuild on the state initiative. 




